AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles

This paper describes the models developed by the AILAB-Udine team for the SMM4H 22 Shared Task. We explored the limits of Transformer based models on text classification, entity extraction and entity normalization, tackling Tasks 1, 2, 5, 6 and 10. The main take-aways we got from participating in different tasks are: the overwhelming positive effects of combining different architectures when using ensemble learning, and the great potential of generative models for term normalization.Comment: Shared Task, SMM4H, Transformer

Extensive Evaluation of Transformer-based Architectures for Adverse Drug Events Extraction

Adverse Event (ADE) extraction is one of the core tasks in digital pharmacovigilance, especially when applied to informal texts. This task has been addressed by the Natural Language Processing community using large pre-trained language models, such as BERT. Despite the great number of Transformer-based architectures used in the literature, it is unclear which of them has better performances and why. Therefore, in this paper we perform an extensive evaluation and analysis of 19 Transformer-based models for ADE extraction on informal texts. We compare the performance of all the considered models on two datasets with increasing levels of informality (forums posts and tweets). We also combine the purely Transformer-based models with two commonly-used additional processing layers (CRF and LSTM), and analyze their effect on the models performance. Furthermore, we use a well-established feature importance technique (SHAP) to correlate the performance of the models with a set of features that describe them: model category (AutoEncoding, AutoRegressive, Text-to-Text), pretraining domain, training from scratch, and model size in number of parameters. At the end of our analyses, we identify a list of take-home messages that can be derived from the experimental data

Monitoring User Opinions and Side Effects on COVID-19 Vaccines in the Twittersphere: Infodemiology Study of Tweets

Background: In the current phase of the COVID-19 pandemic, we are witnessing the most massive vaccine rollout in human history. Like any other drug, vaccines may cause unexpected side effects, which need to be investigated in a timely manner to minimize harm in the population. If not properly dealt with, side effects may also impact public trust in the vaccination campaigns carried out by national governments. Objective: Monitoring social media for the early identification of side effects, and understanding the public opinion on the vaccines are of paramount importance to ensure a successful and harmless rollout. The objective of this study was to create a web portal to monitor the opinion of social media users on COVID-19 vaccines, which can offer a tool for journalists, scientists, and users alike to visualize how the general public is reacting to the vaccination campaign. Methods: We developed a tool to analyze the public opinion on COVID-19 vaccines from Twitter, exploiting, among other techniques, a state-of-the-art system for the identification of adverse drug events on social media; natural language processing models for sentiment analysis; statistical tools; and open-source databases to visualize the trending hashtags, news articles, and their factuality. All modules of the system are displayed through an open web portal. Results: A set of 650,000 tweets was collected and analyzed in an ongoing process that was initiated in December 2020. The results of the analysis are made public on a web portal (updated daily), together with the processing tools and data. The data provide insights on public opinion about the vaccines and its change over time. For example, users show a high tendency to only share news from reliable sources when discussing COVID-19 vaccines (98% of the shared URLs). The general sentiment of Twitter users toward the vaccines is negative/neutral; however, the system is able to record fluctuations in the attitude toward specific vaccines in correspondence with specific events (eg, news about new outbreaks). The data also show how news coverage had a high impact on the set of discussed topics. To further investigate this point, we performed a more in-depth analysis of the data regarding the AstraZeneca vaccine. We observed how media coverage of blood clot-related side effects suddenly shifted the topic of public discussions regarding both the AstraZeneca and other vaccines. This became particularly evident when visualizing the most frequently discussed symptoms for the vaccines and comparing them month by month. Conclusions: We present a tool connected with a web portal to monitor and display some key aspects of the public's reaction to COVID-19 vaccines. The system also provides an overview of the opinions of the Twittersphere through graphic representations, offering a tool for the extraction of suspected adverse events from tweets with a deep learning model

Automatic Verification of Wireless Control in a Mining Ventilation System

International audienceWe address a wireless networked control problem for a mine ventilation system. Ventilation control is essential for the control of the operation of a mine for safety and energy optimization. The main control objective is to guarantee safety of the closed loop system. This test-case is simple enough to be computationally tractable, and yet it exposes the main difficulties encountered when using wireless networked systems for safety-critical applications. The focus of this paper is the formal verification of the operation of a closed loop control system for the so called secondary ventilation system that ensures air flow in the chambers of the mine where extraction takes place. The secondary ventilation system is modeled conservatively in the sense that if the formal verification process provides a positive answer then the system is guaranteed to work correctly while the converse is not necessarily true. For control, we use a simple threshold scheme. The overall closed-loop system is described by a hybrid model that takes into account the effects of time-delay, transmission errors and allows the precise formulation of the safety constraints. To ensure that the formal verification process is computationally tractable, we reason in the framework of temporal logics, and apply abstraction techniques and model checking tools that we developed previously

BERT prescriptions to avoid unwanted headaches : a comparison of transformer architectures for adverse drug event detection

Pretrained transformer-based models, such as BERT and its variants, have become a common choice to obtain state-of-the-art performances in NLP tasks. In the identification of Adverse Drug Events (ADE) from social media texts, for example, BERT architectures rank first in the leaderboard. However, a systematic comparison between these models has not yet been done. In this paper, we aim at shedding light on the differences between their performance analyzing the results of 12 models, tested on two standard benchmarks. SpanBERT and PubMedBERT emerged as the best models in our evaluation: this result clearly shows that span-based pretraining gives a decisive advantage in the precise recognition of ADEs, and that in-domain language pretraining is particularly useful when the transformer model is trained just on biomedical text from scratch

TRY plant trait database – enhanced coverage and open access

Plant traits - the morphological, anatomical, physiological, biochemical and phenological characteristics of plants - determine how plants respond to environmental factors, affect other trophic levels, and influence ecosystem properties and their benefits and detriments to people. Plant trait data thus represent the basis for a vast area of research spanning from evolutionary biology, community and functional ecology, to biodiversity conservation, ecosystem and landscape management, restoration, biogeography and earth system modelling. Since its foundation in 2007, the TRY database of plant traits has grown continuously. It now provides unprecedented data coverage under an open access data policy and is the main plant trait database used by the research community worldwide. Increasingly, the TRY database also supports new frontiers of trait‐based plant research, including the identification of data gaps and the subsequent mobilization or measurement of new data. To support this development, in this article we evaluate the extent of the trait data compiled in TRY and analyse emerging patterns of data coverage and representativeness. Best species coverage is achieved for categorical traits - almost complete coverage for ‘plant growth form’. However, most traits relevant for ecology and vegetation modelling are characterized by continuous intraspecific variation and trait–environmental relationships. These traits have to be measured on individual plants in their respective environment. Despite unprecedented data coverage, we observe a humbling lack of completeness and representativeness of these continuous traits in many aspects. We, therefore, conclude that reducing data gaps and biases in the TRY database remains a key challenge and requires a coordinated approach to data mobilization and trait measurements. This can only be achieved in collaboration with other initiatives

TRY plant trait database – enhanced coverage and open access

Plant traits—the morphological, anatomical, physiological, biochemical and phenological characteristics of plants—determine how plants respond to environmental factors, affect other trophic levels, and influence ecosystem properties and their benefits and detriments to people. Plant trait data thus represent the basis for a vast area of research spanning from evolutionary biology, community and functional ecology, to biodiversity conservation, ecosystem and landscape management, restoration, biogeography and earth system modelling. Since its foundation in 2007, the TRY database of plant traits has grown continuously. It now provides unprecedented data coverage under an open access data policy and is the main plant trait database used by the research community worldwide. Increasingly, the TRY database also supports new frontiers of trait-based plant research, including the identification of data gaps and the subsequent mobilization or measurement of new data. To support this development, in this article we evaluate the extent of the trait data compiled in TRY and analyse emerging patterns of data coverage and representativeness. Best species coverage is achieved for categorical traits—almost complete coverage for ‘plant growth form’. However, most traits relevant for ecology and vegetation modelling are characterized by continuous intraspecific variation and trait–environmental relationships. These traits have to be measured on individual plants in their respective environment. Despite unprecedented data coverage, we observe a humbling lack of completeness and representativeness of these continuous traits in many aspects. We, therefore, conclude that reducing data gaps and biases in the TRY database remains a key challenge and requires a coordinated approach to data mobilization and trait measurements. This can only be achieved in collaboration with other initiatives.Rest of authors: Decky Junaedi, Robert R. Junker, Eric Justes, Richard Kabzems, Jeffrey Kane, Zdenek Kaplan, Teja Kattenborn, Lyudmila Kavelenova, Elizabeth Kearsley, Anne Kempel, Tanaka Kenzo, Andrew Kerkhoff, Mohammed I. Khalil, Nicole L. Kinlock, Wilm Daniel Kissling, Kaoru Kitajima, Thomas Kitzberger, Rasmus Kjøller, Tamir Klein, Michael Kleyer, Jitka Klimešová, Joice Klipel, Brian Kloeppel, Stefan Klotz, Johannes M. H. Knops, Takashi Kohyama, Fumito Koike, Johannes Kollmann, Benjamin Komac, Kimberly Komatsu, Christian König, Nathan J. B. Kraft, Koen Kramer, Holger Kreft, Ingolf Kühn, Dushan Kumarathunge, Jonas Kuppler, Hiroko Kurokawa, Yoko Kurosawa, Shem Kuyah, Jean-Paul Laclau, Benoit Lafleur, Erik Lallai, Eric Lamb, Andrea Lamprecht, Daniel J. Larkin, Daniel Laughlin, Yoann Le Bagousse-Pinguet, Guerric le Maire, Peter C. le Roux, Elizabeth le Roux, Tali Lee, Frederic Lens, Simon L. Lewis, Barbara Lhotsky, Yuanzhi Li, Xine Li, Jeremy W. Lichstein, Mario Liebergesell, Jun Ying Lim, Yan-Shih Lin, Juan Carlos Linares, Chunjiang Liu, Daijun Liu, Udayangani Liu, Stuart Livingstone, Joan Llusià, Madelon Lohbeck, Álvaro López-García, Gabriela Lopez-Gonzalez, Zdeňka Lososová, Frédérique Louault, Balázs A. Lukács, Petr Lukeš, Yunjian Luo, Michele Lussu, Siyan Ma, Camilla Maciel Rabelo Pereira, Michelle Mack, Vincent Maire, Annikki Mäkelä, Harri Mäkinen, Ana Claudia Mendes Malhado, Azim Mallik, Peter Manning, Stefano Manzoni, Zuleica Marchetti, Luca Marchino, Vinicius Marcilio-Silva, Eric Marcon, Michela Marignani, Lars Markesteijn, Adam Martin, Cristina Martínez-Garza, Jordi Martínez-Vilalta, Tereza Mašková, Kelly Mason, Norman Mason, Tara Joy Massad, Jacynthe Masse, Itay Mayrose, James McCarthy, M. Luke McCormack, Katherine McCulloh, Ian R. McFadden, Brian J. McGill, Mara Y. McPartland, Juliana S. Medeiros, Belinda Medlyn, Pierre Meerts, Zia Mehrabi, Patrick Meir, Felipe P. L. Melo, Maurizio Mencuccini, Céline Meredieu, Julie Messier, Ilona Mészáros, Juha Metsaranta, Sean T. Michaletz, Chrysanthi Michelaki, Svetlana Migalina, Ruben Milla, Jesse E. D. Miller, Vanessa Minden, Ray Ming, Karel Mokany, Angela T. Moles, Attila Molnár V, Jane Molofsky, Martin Molz, Rebecca A. Montgomery, Arnaud Monty, Lenka Moravcová, Alvaro Moreno-Martínez, Marco Moretti, Akira S. Mori, Shigeta Mori, Dave Morris, Jane Morrison, Ladislav Mucina, Sandra Mueller, Christopher D. Muir, Sandra Cristina Müller, François Munoz, Isla H. Myers-Smith, Randall W. Myster, Masahiro Nagano, Shawna Naidu, Ayyappan Narayanan, Balachandran Natesan, Luka Negoita, Andrew S. Nelson, Eike Lena Neuschulz, Jian Ni, Georg Niedrist, Jhon Nieto, Ülo Niinemets, Rachael Nolan, Henning Nottebrock, Yann Nouvellon, Alexander Novakovskiy, The Nutrient Network, Kristin Odden Nystuen, Anthony O'Grady, Kevin O'Hara, Andrew O'Reilly-Nugent, Simon Oakley, Walter Oberhuber, Toshiyuki Ohtsuka, Ricardo Oliveira, Kinga Öllerer, Mark E. Olson, Vladimir Onipchenko, Yusuke Onoda, Renske E. Onstein, Jenny C. Ordonez, Noriyuki Osada, Ivika Ostonen, Gianluigi Ottaviani, Sarah Otto, Gerhard E. Overbeck, Wim A. Ozinga, Anna T. Pahl, C. E. Timothy Paine, Robin J. Pakeman, Aristotelis C. Papageorgiou, Evgeniya Parfionova, Meelis Pärtel, Marco Patacca, Susana Paula, Juraj Paule, Harald Pauli, Juli G. Pausas, Begoña Peco, Josep Penuelas, Antonio Perea, Pablo Luis Peri, Ana Carolina Petisco-Souza, Alessandro Petraglia, Any Mary Petritan, Oliver L. Phillips, Simon Pierce, Valério D. Pillar, Jan Pisek, Alexandr Pomogaybin, Hendrik Poorter, Angelika Portsmuth, Peter Poschlod, Catherine Potvin, Devon Pounds, A. Shafer Powell, Sally A. Power, Andreas Prinzing, Giacomo Puglielli, Petr Pyšek, Valerie Raevel, Anja Rammig, Johannes Ransijn, Courtenay A. Ray, Peter B. Reich, Markus Reichstein, Douglas E. B. Reid, Maxime Réjou-Méchain, Victor Resco de Dios, Sabina Ribeiro, Sarah Richardson, Kersti Riibak, Matthias C. Rillig, Fiamma Riviera, Elisabeth M. R. Robert, Scott Roberts, Bjorn Robroek, Adam Roddy, Arthur Vinicius Rodrigues, Alistair Rogers, Emily Rollinson, Victor Rolo, Christine Römermann, Dina Ronzhina, Christiane Roscher, Julieta A. Rosell, Milena Fermina Rosenfield, Christian Rossi, David B. Roy, Samuel Royer-Tardif, Nadja Rüger, Ricardo Ruiz-Peinado, Sabine B. Rumpf, Graciela M. Rusch, Masahiro Ryo, Lawren Sack, Angela Saldaña, Beatriz Salgado-Negret, Roberto Salguero-Gomez, Ignacio Santa-Regina, Ana Carolina Santacruz-García, Joaquim Santos, Jordi Sardans, Brandon Schamp, Michael Scherer-Lorenzen, Matthias Schleuning, Bernhard Schmid, Marco Schmidt, Sylvain Schmitt, Julio V. Schneider, Simon D. Schowanek, Julian Schrader, Franziska Schrodt, Bernhard Schuldt, Frank Schurr, Galia Selaya Garvizu, Marina Semchenko, Colleen Seymour, Julia C. Sfair, Joanne M. Sharpe, Christine S. Sheppard, Serge Sheremetiev, Satomi Shiodera, Bill Shipley, Tanvir Ahmed Shovon, Alrun Siebenkäs, Carlos Sierra, Vasco Silva, Mateus Silva, Tommaso Sitzia, Henrik Sjöman, Martijn Slot, Nicholas G. Smith, Darwin Sodhi, Pamela Soltis, Douglas Soltis, Ben Somers, Grégory Sonnier, Mia Vedel Sørensen, Enio Egon Sosinski Jr, Nadejda A. Soudzilovskaia, Alexandre F. Souza, Marko Spasojevic, Marta Gaia Sperandii, Amanda B. Stan, James Stegen, Klaus Steinbauer, Jörg G. Stephan, Frank Sterck, Dejan B. Stojanovic, Tanya Strydom, Maria Laura Suarez, Jens-Christian Svenning, Ivana Svitková, Marek Svitok, Miroslav Svoboda, Emily Swaine, Nathan Swenson, Marcelo Tabarelli, Kentaro Takagi, Ulrike Tappeiner, Rubén Tarifa, Simon Tauugourdeau, Cagatay Tavsanoglu, Mariska te Beest, Leho Tedersoo, Nelson Thiffault, Dominik Thom, Evert Thomas, Ken Thompson, Peter E. Thornton, Wilfried Thuiller, Lubomír Tichý, David Tissue, Mark G. Tjoelker, David Yue Phin Tng, Joseph Tobias, Péter Török, Tonantzin Tarin, José M. Torres-Ruiz, Béla Tóthmérész, Martina Treurnicht, Valeria Trivellone, Franck Trolliet, Volodymyr Trotsiuk, James L. Tsakalos, Ioannis Tsiripidis, Niklas Tysklind, Toru Umehara, Vladimir Usoltsev, Matthew Vadeboncoeur, Jamil Vaezi, Fernando Valladares, Jana Vamosi, Peter M. van Bodegom, Michiel van Breugel, Elisa Van Cleemput, Martine van de Weg, Stephni van der Merwe, Fons van der Plas, Masha T. van der Sande, Mark van Kleunen, Koenraad Van Meerbeek, Mark Vanderwel, Kim André Vanselow, Angelica Vårhammar, Laura Varone, Maribel Yesenia Vasquez Valderrama, Kiril Vassilev, Mark Vellend, Erik J. Veneklaas, Hans Verbeeck, Kris Verheyen, Alexander Vibrans, Ima Vieira, Jaime Villacís, Cyrille Violle, Pandi Vivek, Katrin Wagner, Matthew Waldram, Anthony Waldron, Anthony P. Walker, Martyn Waller, Gabriel Walther, Han Wang, Feng Wang, Weiqi Wang, Harry Watkins, James Watkins, Ulrich Weber, James T. Weedon, Liping Wei, Patrick Weigelt, Evan Weiher, Aidan W. Wells, Camilla Wellstein, Elizabeth Wenk, Mark Westoby, Alana Westwood, Philip John White, Mark Whitten, Mathew Williams, Daniel E. Winkler, Klaus Winter, Chevonne Womack, Ian J. Wright, S. Joseph Wright, Justin Wright, Bruno X. Pinho, Fabiano Ximenes, Toshihiro Yamada, Keiko Yamaji, Ruth Yanai, Nikolay Yankov, Benjamin Yguel, Kátia Janaina Zanini, Amy E. Zanne, David Zelený, Yun-Peng Zhao, Jingming Zheng, Ji Zheng, Kasia Ziemińska, Chad R. Zirbel, Georg Zizka, Irié Casimir Zo-Bi, Gerhard Zotz, Christian Wirth.Max Planck Institute for Biogeochemistry; Max Planck Society; German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig; International Programme of Biodiversity Science (DIVERSITAS); International Geosphere-Biosphere Programme (IGBP); Future Earth; French Foundation for Biodiversity Research (FRB); GIS ‘Climat, Environnement et Société'.http://wileyonlinelibrary.com/journal/gcbhj2021Plant Production and Soil Scienc

Personal Transportation Energy Consumption

This paper centers on the estimation of the total energy consumption for personal transportation in the United States, to include fossil fuel and/or electricity consumption, depending on vehicle type. The bottom-up sector-based estimation method introduced here contributes to a computational tool under development at The Ohio State University for assisting decision making in energy policy, pricing, and investment. In this work, driving patterns are classified into two categories: commuting to work, and driving for leisure and shopping. For commuting, distribution of distance data is available in the literature. Leisure/shopping driving durations are estimated using activity patterns for a driving population, modeled using a heterogeneous Markov chain. A backward vehicle dynamic simulator is used to compute energy consumption for different vehicle types. Key findings of the current study include: (i) Independent of the total number of miles driven annually, the higher the vehicle electrification the lower the total primary energy consumption. (ii) With the modeling in this work, the percentage of trips that purely electric vehicles are unable to complete varies from 7% to 13% for driving distances up to 20000 miles per year. The percentage increases significantly for driving distances over that threshold, owing to intrinsic limitations of the battery